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GUIDANCE ON SUBJECTIVE 
LISTENING TESTS 

0. FOREWORD 

0.1 This Indian Standard was adopted by the Indian Standards Institution 
on 28 July 1981, after the draft finalized by the Radio Communications 
Sectional Committee had been approved by the Electronics and Tele- 
communication Division Council. 

0.2 This standard is largely based on IEC Pub 543(1976) 'Informative 
guide for subjective listening tests' issued by the International Electro- 
technical Commission (IEC). 

0.3 For the purpose of deciding whether a particular requirement of this 
standard is complied with, the final value, observed or calculated, express- 
ing the result of a test, shall be rounded off in accordance with IS : 2-1960*. 
The number of significant places retained in the rounded off value should 
be the same as that of the specified value in this standard. 



1. SCOPE 

1.1 This standard gives informative guidance regarding subjective listening 
tests on radio receivers. 

2. PRINCIPLE OF SUBJECTIVE LISTENING TESTS 

2.1 A need exists for reproducible subjective listening tests, because 
objective measurements alone have not been found adequate for determin- 
ing sound quality. However, as subjective tests have an inherent degree of 
uncertainty, objective tests should still be used wherever possible, particularly 
for those parameters that are to be included in the terms of a contract. 

2.2 The purpose of this guide is to define procedures that will permit com- 
parison of the acoustical performance of various types of household or other 
similar sound radio receivers, in a controlled environment and with an 
established procedure, to give results that are as reproducible as possible. 
The listening tests are carried out by comparison against a reference 
receiver. The reference receiver shall be selected with great care in all cases, 
because the comparative judgements made are dependent on this reference. 
A description of the reference receiver shall also be given with the results. 

♦Rules for rounding off numerical values ( revised)- 
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2.3 A subjective evaluation, based on this guide, consists of the average 
judgement of several persons on a given subject. The individual opinions 
are divided into ranges corresponding to assessment scales, showing for 
each range the number of persons of that opinion. In this way, the subjec- 
tive evaluation may be presented as an average opinion with a standard 
deviation. 

2.4 Receivers should be divided into categories of use for comparison so 
that each is judged within its own category. 

Such categories might be, for example: 

~) Al AM personals 
A : personals ( pocket receivers ) y A2 FM personals 

j A3 AM/FM personals 

] Bl AM portables 
B : P ° rtabIeS i B2 AM/FM portables 



C ; table sets for A.M. 



} 



CI table sets for AM 
C2 table sets for AM/FM 



D : other receivers ( for example, larger radiograms ) 
E : High-fidelity equipment. 

3. CHARACTERISTICS OF THE TEST ROOM 

3.1 For consumer testing, a room of the same type as a living room with 
the average characteristics of domestic surroundings is advisable. 

3.1.1 It is advisable to use a room of the same type as a living room in 
which the shortest side should be not less than 3'5ra (see Appendix A ) in 
order to be able to measure stereophonic equipment (stereo basis >2'5m). 

3.2 The acoustic response of the room shall be fairly smooth and the 
reverberation time shall be that of a typical living room. 

3.2.1 This time is approximately 0*5±0'1 s at 500 Hz (inclusive of 
panelists ). 

3.2.2 Over the frequency range 100—5 000 Hz there shall not be any 
unusual room resonances nor flutter echo. 

3.3 The lighting within the room shall be of medium intensity, so arranged 
that the lights do not shine directly into the eyes of any panelists. 

3.3,1 Temperature shall be in the range 15° to 35°C. 
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3.3.2 Relative humidity shall be in the range 45 to 75 percent. 

Note— Damping elements consisting of domestic furniture, curtains and carpets 
are to be preferred to sound absorbing panels in ensuring the correct relationship 
between reverberation time and frequencies. 

Rooms with solid floors (covered with pile carpet) and without large areas of 
glazing are easier to correct. 

3.4 The recommended background sound level is less than 35 dB (A) 
determined with a sound level meter. If this level is exceeded, the level 
shall be stated with the results. 

3.5 The volume of the reference equipment and that of the receiver under 
test shall be adjusted to a level acceptable to the listening panel. 

3.6 As purely indicative, sound levels for 'medium' receivers of 75 dB (A) 
are suitable for listening in living room, that is, the sound level for 
domestic purposes shall be as follows: 

a) for small receivers approximately 60 dB (A) 

b) for medium receivers 75 dB (A) 

c) for large and Hi-Fi receivers 80-85 dB (A) 

( see also 10.1) 

4. NUMBER OF PANELISTS 

4.1 A representative sample of observers for comparison for a statistically 
high level of confidence is twenty or more. 

4.2 For most purposes, the number of panelists may be reduced to seven 
( or even less ); such panels have been found to give reproducible results, 
if precautions are taken as specified in 5 and 11. 

5. QUALIFICATION OF PANELISTS 

5.1 The panel shall be a representative sample of users of the particular 
type of equipment under test. 

5.2 Consequently, the matters to be considered are as follows: 

a) experts/laymen, 

b) degree of musical appreciation, 
cj age, 

d) sex, and 

e) hearing ability. 



IS: 9887 -1981 

5.3 In general, an appropriate audio test should make it possible to verify 
that the listeners do not have a very reduced hearing ability. 

5.4 In certain cases it may be convenient to select panelists in a particular 
manner, especially when the panel has to be small. 

5.5 Such a selection may be made in different ways, for example, one may 
use the personal standard deviation derived from previous tests, or by an 
audiogram that indicates the following: 

a) the perception of sound level differences of approximately 3 dB and 
change of pitch of 1 percent at 1 000 Hz; 

b) a sensitivity of hearing in the frequency range up to 7 kHz not 
worse than the appropriate value indicated in the diagrams in 
Appendix B, and 

c) the ability to place accurately the image position of a 'pink noise' 
( see Note 2 ) signal source fed to a pair of loudspeakers, set up in 
the normal stereo fashion. 

Note 1 — For table radios, portable radios and low and medium priced gramo- 
phones, the panel of listeners need not be musically educated, but they should 
be musically perceptive. 

For high quality radios and gramophones, 'high-fidelity system', the procedures is 
generally similar, except that at least half the panel shall be musically educated. 

Note 2 — 4 Pink noise' is oscillation whose spectrum is uniform as a function of 
frequency, over a designated frequency range, as measured in a band width 
proportional to the frequency. 

5.6 As far as possible, the composition of the panel shall be such that the 
number of men and women is equal in each age group as shown in Table 1. 

5.7 With a higher number of panelists repeated tests shall be made 
maintaining the ratios given in Table 1 . 

5.8 In any case, the composition of the panel shall be stated with the results. 

6. ARRANGEMENT OF THE RECEIVERS 

6.1 The receivers under test shall not be visible to the panel ( this is to 
avoid psychological influences ) and shall be placed behind a sound trans- 
parent curtain. 

6.2 It is suggested that the layout of the test room should be in the general 
form shown in the diagram in Appendix A ( the illustration is for both 
stereophonic and monophonic equipment as appropriate ). 
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TABLE 1 SUGGESTED COMPOSITION OF THE PANEL 

( Clause 5.6 ) 

Number of Panelists 







Minimum 


Recommended 


Maximum 


Sex 












Men 


2 


3 or 4 


5 




Women 

Total 


2 


4 or 3 


5 




4 


7 


10 


Age 










Between 15 and 40 


2 


4 


5 




Between 40 and 60 


1 


2 


3 




Over 60 

Total 


1 


1 


2 




4 


7 


10 











6.3 The microphone ( M ) of a sound level meter is suspended among the 
panelists above head level. 

6.4 As far as possible, loudspeakers of the stereophonic equipment under 
test shall be placed at such distance ( D) apart and from the listeners as are 
recommended in the manufacturer's instructions, but in no case shall 
listener 1 -be at a distance ( H ) less than 2*5 m from the curtain. 

6.5 This arrangement is also satisfactory for monophonic equipment, but 
not for the case of pocket radio. 

6.6 A monophonic radio is placed in the centre and just behind the sound 
curtain. 

6.7 All loudspeakers other than those intended to stand on the floor shall 
be placed on a solidly constructed wooded table, 750-800 mm in height, 
such that the loudspeakers are at the same distance from the edge of the 
table nearest to the panel. 

6.8 When testing pocket radios, the panelists shall hold the receivers in 
their hands, covered if possible. 
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7. REFERENCE EQUIPMENT 

7.1 It is essential to make the assessment by comparison with a reference 
receiver, preferably a sample which is of a quality corresponding to the 
characteristic under consideration and near to the centre of the grading 
scale. 

7.2 In addition, a receiver of the highest quality may be used as evidence 
of achievable sound quality. 

7.3 Consistency between different series of evaluations may be achieved 
using the same test equipment for comparison. 

8. SIGNAL INPUT 

8.1 A predetermined programme according to 9 shall be recorded on and 
reproduced by means of a high quality tape deck. 

8.2 The audio frequency output from the tape deck shall be fed to a signal 
generator and thence as a radio frequency signal ( AM or FM ) fed to the 
radio receivers under test, by means of a radio loop or by suitable coupling 
means, for example, a magnetic aerial or as a voltage via the proper aerial 
impedance network. 

8.3 The level of the signal shall have an appropriate value and be substan- 
tially the same for each receiver. For receivers with audio frequency input 
the tape deck output is fed through a matching network to this input in a 
separate test. 

8.4 The characteristics of the signal, especially the depth of modulation in 
the case of AM and the frequency deviation and the pre-emphasis charac- 
teristics in the case of FM, shall correspond closely to the transmission 
standard used. 

8.5 The levels of the input signals used shall be stated with the results. 

9. CHARACTERISTICS OF THE MODULATION SIGNALS 

9.1 The programme shall include the following: 

a) Classical music ( a symphonic piece, a chamber music piece and an 
operatic song ). 

b) Popular music ( light rhythmical music of all type, including light 
classical, dance, and jazz, with and without vocals ). 

c) Speech in different languages. 
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9.2 In the case of vocal transmission of a text, the speaker shall have an 
excellent diction. Suggested duration of each type vary approximately as 
follows: 



a) for pink noise 


1 min 


b) for classical music 


3 min 


c) for light music 


: 6 min 


d) for speech 


: 3 min 



9.3 Each test piece shall not last more than 20s and shall be played through 
the reference equipment immediately before being played through the 
equipment under test. 

9.4 This procedure is to be repeated for each test piece and for each 
receiver tested. 

9.5 A switchboard shall be used, so that quick changes between the 
reference receiver and test receivers are possible ( with period of silence 
about 0-5s ). 

10. PROCEDURE 

10.1 Adjustment of Volume Controls 

10.1.1 The tests shall be conducted using the same volume level for the 
reference receiver and the receiver (s) under test as determined by the sound 
level meter referred to in 6 using a 'pink noise' modulation. 

10.1.2 Volume has to be chosen according to the type of the test 
programme [ about 75 dB ( A ) for 'medium' receivers, about 60 dB ( A ) 
for 'small' receivers, and 80-85 dB ( A ) for 'Hi-Fi' equipment ]. 

10.1.3 The position and height at which the sound pressure levels are 
measured shall be stated with the results. It is desirable to use for the 
measurement the level meter having the slow time constant of weighting 
curve A prescribed in IS: 3932-1966*. 

10.1.4 A short test shall also be made at high volume setting to indicate 
any unwanted resonance or other forms of distortion both electrical and 
mechnical. 

10.2 Adjustment of Tone Controls 

10.2.1 The controls for treble and bass shall be set to linear frequency 
response. As far as possible, tests may also be made with other setting of 
the tone control and this shall be recorded with the results. 

10.2.2 For table of minimum checks to be made ( see Appendix C ). 



♦Specification for sound level meter for general purpose use. 
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10.3 Evaluation 

10.3.1 The total sound impression ( overall sound quality ) and the 
intelligibility of speech shall be evaluated. 

10.3.2 In addition to or as part of these tests the panel may be asked to 
evaluate, for example: 

a) the reproduction of low notes and balance between the two 

b) the reproduction of high notes 

c) the absence of harshness 

d) the absence of hum 

e) the absence of distortion 

f ) the absence of crackle and other undesirable noise 

g) the reproduction of transients. 

10.3.3 For the judgement of the stereophonic effect, a specially prepared 
tape shall be used in relation to which the position of the instruments 
across the stage is known. 

10.3.4 Separate judgements shall be recorded for AM and FM. 

10.3.5 The intelligibility of speech, if necessary, may be expressed as a 
percentage of the number of words understood in relation to the number of 
words transmitted, when using a special series of words. 

10.3.6 Alternatively, the judgement may be expressed by the panel in 
terms of the scales in 13.2, when using various types of normal speech. 

10.3.7 The judgement of total sound impression shall be the final 
evaluation. 

11. CONDITIONS OF WORK AND INSTRUCTIONS TO THE PANEL 

11.1 The panel shall not be asked to make judgements for more than 
20 min without a break, and the break shall be at least equal in time to the 
time they have marked. 

11.2 A session shall not last more than 2 h, including both working 
periods and breaks. 

Note — The persons in charge of the tests shall ensure that the questions addressed 
to the panel are clear and simple. 

10 
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The panel shall be carefully instructed before the start of each test 
session concerning: 

a) the category of the receiver being tested, 

b) the way in which the test will be conducted, and 

c) the method of scoring to be used. 

12. MAINS INPUT AND STABILITY CONDITIONS 

12.1 Signal strength and mains input shall not be varied, and interfering 
signals shall not be perceptible during normal listening tests. 

12.2 However, the effects of interference may be evaluated by a listening 
panel, if required. 

13. SCORING AND SCALES OF SUBJECTIVE GRADINGS 

13.1 The scoring shall be considered for every item and the final scores of 
each item obtained expressed in terms of a mean opinion and the corres- 
ponding standard deviation. 

13.1.1 Scoring is to be made for each receiver and each example of 
programme. 

Note — The mean value and the standard deviation are calculated as follows: 
Mean value, P m — Z n 



Standard deviation, s 



V n-l 



where P t Pi P n are the n different marks obtained. 

Should there be a wide dispersion among the panelists, these parts of the 
test in question may possibly have to be repeated and the panelists be asked 
to evaluate it again. 

The score to be used by each panelist for each receiver and each section of 
the test is as follows. 

11 
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13.2 Scales of Evaluation 

13.2.1 Comparative grades* are as follows: 



— 2 : much worse = 


2 points 


— 1 : worse = 


4 points 


: same as = 


6 points 


+ 1 : better = 


8 points 


+2 : much better = 


10 points 


13.2.2 Absolute grades are as follows: 




Impairment 


Quality 


5 : imperceptible 


5 : excellent 


4 : perceptible but not annoying 


4 : good 


3 : perceptible, slightly annoying 


3 : fair 


2 : annoying 


2 : poor 


1 : very annoying 


1 : bad 



13.2.3 Either a comparison or an absolute classification -shall be used as 
appropriate. 

Note— It is recognized that to suit local use, some laboratories may wish to replace 
the numbers with letters or points. 

However, it is proposed that the results shall be converted to the 
form given above. 



*This comparison scale is a simplified version of the 7 point scale now recommended 
by CCIR for sound- 

12 
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APPENDIX A 

( Clauses 3.1.1 and 6.2 ) 

LAYOUT OF THE TEST ROOM 



56m 




AC = ARTIFICIAL 

CORNER 
(WHEN REQUIRED*} 



DISTANCE A SHOULO 
BE STATED WITH 
THE RESULTS 



Fig. 1 



*Note — The construction of the artificial corner is most important, since most 
light-weight partitioning will cause unfair bass loss: the transmission attenuation 
shall be at least 20 dB, down to 100 Hz. It may be made up from a rectangular wooden 
framework constructed from 50 cm X 25 cm timber on both sides of which is fixed 
plaster board of 2 cm thickness, so that the internal separation of the boards is 5 cm- 
The intervening air space is loosely filled with fibre glass. 

13 
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APPENDIX B 

[ Clause 5.5 (b) ] 

HEARING LOSS FREQUENCY CHARACTERISTICS OF MEN 

AND WOMEN FOR DIFFERENT AGE GROUPS 



MEN 




860 1760 3520 

FREQUENCY IN Hz 



WOMEN 




1760 3520 

FREQUENCY IN Hz 



Fig. 2 
14 



APPENDIX C 

( Clause 10.2.2 ) 
TABLE OF MINIMUM CHECKS TO BE MADE 



Receiver 
Category* 


Level Adjustment 


Symphony 
Orche- 
stra 


Organ 


Solo 


Song 

Soprano 


Speech Light Music 


Volume 


Tone 


I 


II 


Dance 
Band 


Jazz 
Band 


A ... D 


normal! 


see] 10,2 


X 


X 


X 


X 


X 


X 


X 


X 


E 

( Hi-Fi } 


normalf 


linear 

frequency 

response 


X 


X 


X 


X 


X 


X 


X 


X 


A... D 


normaif 


linear 

frequency 

response 










X 






X 


E 
( Hi-Fi ) 


norrnalf 


see 10.2 










X 






X 


A... E 


high 
volume^ 


see 10.2 


X 








X 









*See Foreword. 

\See 10.1 — approximate 75 dB(A) [ small receivers approximate 60 dB(A) J. 

tSee 10.1 — approximate 80 dB to 85 dB(A). 
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